1. Field of the Invention
The present invention relates to a device for detecting the line of sight of the observer, utilizing a reflected image of the eyeball obtained by illuminating the eyeball surface of the observer.
2. Related Background Art
There have been proposed various devices for detecting the position in the viewing field, observed by the observer, or so-called line of sight thereof, such as the eye camera.
For example, the U.S. Ser. No. 327,784 discloses a device for projecting a parallel light beam from a light source to the frontal part of the eyeball of the observer, and determining the line of sight by means of the corneal reflected image and the image position of the pupil, formed by the light reflected from the cornea.
FIGS. 10A, 10B and 11 show the principle of detection of the line of sight.
In the following there will be given an explanation on a sight line detecting device, applied to a camera.
At first referring to FIG. 11, infrared light-emitting diodes (IRED's) 13a, 13b are positioned in the x-direction substantially symmetrically with respect to the optical axis i of a light-receiving lens 12, and respectively illuminate the eyeball of the photographer in diffuse manner.
The infrared light emitted from the IREDs 13a, 13b illuminates the cornea 16 of the eyeball 15. Corneal reflected images d, e formed by a part of the infrared light reflected at the surface of the cornea 16, are refocused by the light-receiving lens 12 at positions d', e' on an image sensor 14.
Also the image of the pupil of the eyeball illuminated by the IRED's is formed on the image sensor 14. The center C of the circular boundary between the pupil and the iris (said boundary being called pupil circle), having an x-coordinate x.sub.c, has an unrepresented x-coordinate x.sub.c ' on the image sensor 14.
FIG. 10A shows the eyeball image projected onto the image sensor 14 shown in FIG. 2, and FIG. 10B shows the image signal output along a line (I)-(I') in FIG. 10A.
In FIG. 10A there are shown so-called white 50 of the eyeball, a pupil 51, and corneal reflected images 52a, 52b of a pair of IREDs.
Said corneal reflected image is called Purkinje's image.
In the signal 60 shown in FIG. 10B, two maxima correspond to the paired Purkinje's images.
Referring again to FIG. 11, since the x-coordinate of the center of the corneal reflected images d, e coincides with the x-coordinate x.sub.0 of the center O of curvature of the cornea 16, the rotation angle .theta. of the optical axis k of the eyeball approximately satisfies a relation: EQU (A1*L.sub.0C)* sin .theta..congruent.xc-(xd+xe)/2 (1)
wherein Xd, Xe are X-coordinates of the positions d, e where the corneal reflected images are generated, L.sub.OC is a standard distance from the center O of curvature of the cornea 16 to the center C of the pupil 19, and Al is a coefficient representing individual fluctuation on said distance L.sub.OC. Consequently, in a sight line calculating device, the rotation angle .theta. of the optical axis k of the eyeball can be determined by detecting the positions of feature points (corneal reflected images d, e and center C of the pupil) projected on the image sensor. In this operation, the equation (1) is re-written as: EQU .beta.(A1*L.sub.0C)*sin .theta..congruent.xc'-(xd'+xe')/2 (2)
wherein .beta. stands for a magnification determined by the position of the eyeball with respect to the light-receiving lens 12, and is practically determined as a function of the distance .vertline.xd'-xe'.vertline. of the corneal reflected images. Also the rotation angle 74 of the eyeball 15 is re-written as: EQU .theta..congruent.ARCSIN {xc'-xf')/.beta./ (A1 * L.sub.0C)}(3)
wherein: EQU xf".congruent.(xd'+xe')/2.
Since the optical axis k of the eyeball of the photographer does not coincide with the line of sight, the line .theta.H of sight of the photographer in the horizontal direction can be determined by an angular correction .delta. between the optical axis of the eyeball and the line of sight, once the rotation angle .theta. of the optical axis k of the eyeball in the horizontal direction is calculated. Taking a coefficient B1 for the individual fluctuation for the correction angle .delta. between the optical axis k of the eyeball and the line of sight, line .theta.H of sight of the photographer in the horizontal direction can be given by: EQU .theta.H=.theta..+-.(B1*.delta.) (4)
wherein the sign .+-. is + or - respectively if the photographer looks at the device with the left eye or the right eye, when the rotation angle to the right with respect to the photographer is taken as positive.
FIG. 11 shows the case of the rotation of the eyeball of the photographer in the Z-X plane (for example horizontal plane), but the detection is similarly possible also in case of rotation of the eyeball in the Z-Y plane (for example vertical plane). However, since the vertical component of the line of sight of the photographer coincides with the vertical component .theta.' of the optical axis of the eyeball, the line of sight .theta.V in the vertical direction is represented by: EQU .theta.V=.theta.'.
Based on the sight line data .theta.H and .theta.V, the coordinates (Xn, Yn) looked at by the photographer on the focusing screen in the viewfinder field is given by: ##EQU1## wherein m is a constant determined by the finder optical system of the camera.
The coefficients A1, B1 for correcting the individual fluctuation of the eyeball of the photographer can be determined by letting the photographer watch an index provided in a predetermined position in the viewfinder and matching the position of the watched point calculated according to the equation (5) with the position of said index.
The calculation for determining the line of sight of the photographer and the watched point is executed by the software of a microcomputer of the sight line processing device, according to the foregoing equations.
After the determination of said coefficients for correcting the individual difference in the line of sight, the position, on the focusing screen, of the line of sight of the photographer looking at the viewfinder is calculated according to the equation (5), and thus obtained information on the line of sight is utilized for focusing control of the phototaking lens or for exposure control.
The actual determination of the line of sight is achieved by detecting the Purkinje's image and the pupil circle mentioned above by processing the eyeball image on the image sensor with the microcomputer and utilizing the positional information thus obtained.
A specific method is already proposed in the Japanese Patent Application No. 3-121097. According to this method, the pupil circle is determined by reading the eyeball image signal from the image sensor, extracting the difference in luminance at the boundary between the pupil and the iris as signal edge, memorizing the coordinates of such signal edges, and, after the readout of the eyeball image, estimating a circle from the plural coordinates of the pupil edge by the least square method.
FIG. 12A shows an eyeball image, in which the Purkinje's image is omitted. Plural white circles positioned around the pupil 51 represent pupil edge, and a numeral 70-1 stands for one of such edge positions.
FIG. 12B shows the pupil edge shown in FIG. 12A, and dots in an area 71 are those extracted as the upper edge of the pupil circle. Also areas 72, 73, 74 respectively show the lower, left, and right edges.
A circle 75 is estimated by the least square method from these edge data, as shown in FIG. 12C, and has coordinates (xc, yc) of the center and a radius r.sub.c.